Enhancing Referential Success by Tracking Hearer Gaze
نویسندگان
چکیده
The ability to monitor the communicative success of its utterances and, if necessary, provide feedback and repair is useful for a dialog system. We show that in situated communication, eyetracking can be used to reliably and efficiently monitor the hearer’s reference resolution process. An interactive system that draws on hearer gaze to provide positive or negative feedback after referring to objects outperforms baseline systems on metrics of referential success and user confusion.
منابع مشابه
Effects of Cognitive Effort on the Resolution of Overspecified Descriptions
Studies in referring expression generation (REG) have shown different effects of referential overspecification on the resolution of certain descriptions. To further investigate effects of this kind, this article reports two eye-tracking experiments that measure the time required to recognize target objects based on different kinds of information. Results suggest that referential overspecificati...
متن کاملSpeaker gaze affects utterance comprehension beyond visual attention shifts
Previous research has shown that listeners follow speaker gaze to mentioned objects in a shared visual environment to ground referring expressions, both for human and robot speakers. What is less clear is whether listeners exploit speaker gaze to infer referential intentions (Staudte & Crocker, 2010), or whether the benefits of gaze can be more simply explained by (reflexive) gaze following (Fr...
متن کاملDogs' Gaze Following Is Tuned to Human Communicative Signals
Recent evidence suggests that preverbal infants' gaze following can be triggered only if an actor's head turn is preceded by the expression of communicative intent [1]. Such connectedness between ostensive and referential signals may be uniquely human, enabling infants to effectively respond to referential communication directed to them. In the light of increasing evidence of dogs' social commu...
متن کاملOn the importance of gaze and speech alignment for efficient communication
Gaze is known to be an important social cue in face-to-face communication indicating focus of attention. Speaker gaze can influence object perception and situated utterance comprehension by driving both interlocutors’ visual attention towards the same object; hence facilitating grounding and disambiguation [1]. The precise temporal and causal processes involved in on-line gaze-following during ...
متن کاملInvestigating joint attention mechanisms through spoken human-robot interaction.
Referential gaze during situated language production and comprehension is tightly coupled with the unfolding speech stream (Griffin, 2001; Meyer, Sleiderink, & Levelt, 1998; Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995). In a shared environment, utterance comprehension may further be facilitated when the listener can exploit the speaker's focus of (visual) attention to anticipate, groun...
متن کامل